[SYSTEMDS-3899] Unary out-of-core operations#2298
Conversation
Heavy hitter instructions: # Instruction Time(s) Count 1 ooc_rblk 0.055 1 2 toString 0.049 2 3 print 0.016 2 4 createvar 0.013 3 5 ooc_ceil 0.001 1 6 rmvar 0.000 3
| runTest(true, false, null, -1); | ||
|
|
||
| HashMap<MatrixValue.CellIndex, Double> dmlfile = readDMLMatrixFromOutputDir(OUTPUT_NAME); | ||
| // Double result = dmlfile.get(new MatrixValue.CellIndex(1, 1)); |
There was a problem hiding this comment.
some check for result correctness would be good. You might use an aggregation and compare just the final scalar value.
|
|
||
| # Read input matrix and operator from command line args | ||
| X = read($1); | ||
| print(toString(X)) |
There was a problem hiding this comment.
avoid because currently, we only support a single consumer on a streaming OOC input. The toString would collect the entire matrix.
| Y = ceil(X); | ||
| print(toString(Y)) | ||
| # Write the final matrix result | ||
| write(Y, $2, format="binary"); |
There was a problem hiding this comment.
the write is currently not yet supported so better use an aggregation and write the result as.matrix(agg)
There was a problem hiding this comment.
done. I've used sum aggregation and tested it against Java result.
|
Hi @mboehm7 , thanks for the review. I've addressed the comments. |
|
LGTM - Thanks for the quick improvements @j143. During the merge I only slightly improved the test via (setExecMode for consistent handling). |
2 participants
this one focuses on
ceil(X)operation.